name: Charlie Belinky

Introduction

The source for my dataset is: https://www.kaggle.com/andrewmvd/doom-crossing?select=doom_crossing_dataset.csv. This dataset contains 1597 image posts extracted from both r/doom and r/animalcrossing, including metadata. Images and metadata come from the source reddit post including post title, upvotes, downvotes, time of post creation, url, etc.

The target is the actual image. I plan to use this dataset to train models that are able to differentiate an image from the Animal Crossing and Doom subreddits.

Setup

Random sample of images from each dataset below

Analysis:

These images are taken from respective subreddits from animal crossing and doom eternal (r/doom and r/animalcrossing). This means that the datasets not only include screenshots of both games, but also memes that may contain little to no assets related to either game in the image.

Some images are only text, which is more difficult to classify.

Some memes formats are used on both subreddits.

Some memes about Animal Crossing include refrences to Doom and vice versa, which can make classification difficult.

Neural Network (Keras)

Since I was not given a train and test set of images from the Kaggle source, I made my own from the directory of images. In total, the 'animal_crossing' and 'doom' image datasets contained a combined 1,680 images. I decided to take 1000 images in total for the train sets, and the remaining 597 images for the test set. This left me with the following number of images in each set:

train

ac- 459 images

doom- 541 images

test

ac- 298 images

doom- 299 images


code source https://www.geeksforgeeks.org/python-image-classification-using-keras/

Analysis:

The Keras model was successful. Using 10 epochs, took about 30 minutes to run the model to a point where the accuracy was about 78%. That is an increase in 20% accuracy from epoch 1 to epoch 10. I believe the accuracy would continue to increase if I added more epochs. Neural network models are the model I predict will be the most successful when classifying images.

Neural Network (Pytorch)

Image Pre-processing

Using PyTorch to classify images requires the images to be transformed into tensors in order to train the model. Here the images from the dataset are standardized in pixel intensity and dimensions and transformed into tensors.


Code source: https://www.pluralsight.com/guides/image-classification-with-pytorch

Analysis:

My Pytorch model was not successful. I was interested to see how it compared to the other neural network model, Keras, however it was much harder to implement. I found a solution online (cited above), that helped me attempt to use a Pytorch model, however my attempts were unsuccessful. In theory, Pytorch should outperform the Keras model in either accuracy or speed.

XGBoost

Once again, I split the given dataset into training and testing datasets using the same method as before resulting in datasets as follows:

train

ac- 459 images

doom- 541 images

test

ac- 298 images

doom- 299 images


source: https://youtu.be/2miw-69Xb0g

Analysis:

To my suprise, the XGBoost model resulted in 100% accuracy. After running a confusion matrix, and a simple test that displays a random image, the predicted value, and the actual value, everything checks out. I do not believe it is actually realistic that a model can classify images with 100% accuracy, so something in the way the data was prepared for feature extraction must not be correct, hoewever I can't figure out what it is.

Decision Tree Classification

source code: https://www.datacamp.com/community/tutorials/decision-tree-classification-python

Analysis:

classification rate of 100%, which does not seem possible! I used the same method of creating and formatting input data as I did when I ran the XGBoost model (which also got 100% accuracy), leading me to believe that there is some sort of flaw with data input. Whether it be mis-classification or some sort of give away within the data as to what category an image belongs to.

Logistic Regression

Since this project is about image classification, I decided that logistic regression was not a good fit. When using regression, the output variable is expected to be a continuous numerical value. With that being said, log regression is used for solving classification problems despite being a form of regression. With log regression, the target variable becomes categorical since it will have a binary output (classified as either 0 or 1). Using the Sigmoid function (which has values both very close to 0 and very close to 1 on the 'S'-shaped line), a model can predict whether or not an input is 0 or 1 depending on the threshold that is determined.

Logisitc regression is no the best fit for this project simply because neural networks are so much better for image classification. In a sense, log regression is basically a single-layer neural network. With that in mind, log regression would make for a similar, but less accurate CNN model.

Conclusion

Overall, between all of the successful methods, XGBoost and the decision tree classification were the most accurate. With that being said, both of those methods had an accuracy of 100% which is a bit unrealistic to me, leading me to believe that there was some underlying data that gave away the correct label for any given image. If I had more time, I would try and figure out what might be causing the accuracy to be so high. Lastly, the keras neural network reached an accuracy of about 78% which seemed much more realistic. The data input method for keras was much more straight foward, and much easier to implement leaving me with less room for error and possibly giving me support for my claim that the other 2 successful methods had some sort of fault during data input. The keras neural network also took the longest time to complete at around 30 minutes for 10 epochs, and given more time to train the model on more epochs, the accuracy would have increased.